Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Store everest results in ERT storage #9161

Open
wants to merge 7 commits into
base: main
Choose a base branch
from

Conversation

yngve-sk
Copy link
Contributor

@yngve-sk yngve-sk commented Nov 6, 2024

Issue
Resolves #8811

Store it as some dataframes and dump it in storage, not entirely sure how much of it should be saved. Will use everest_data_api as reference for what is required.

Objectives, constraints and control names are pivoted so that each distinct value has its own column, which is more concise and better to serve via an API. It makes each row in the dataset represent the entire set of controls, objectives, and constraints for one batch | batch -> realization | batch -> realization -> perturbation.

objectives_for_batch.to_dicts()
Out[5]: 
[{'result_id': 0,
  'batch_id': 0,
  'total_objective_value': 1886.010009765625,
  'npv_function': 1886.010009765625}]
objectives_per_real.to_dicts()[:4]
Out[6]: 
[{'batch_id': 0,
  'realization': 0,
  'npv_function': 1886.010009765625,
  'group_w14': 0.0623,
  'group_w01': 0.0627,
  'group_w12': 0.0621,
  'group_w09': 0.0618,
  'group_w03': 0.0629,
  'group_w13': 0.0622,
  'group_w07': 0.0633,
  'group_w04': 0.063,
  'group_w02': 0.0628,
  'group_w10': 0.0619,
  'group_w08': 0.0617,
  'group_w05': 0.0631,
  'group_w11': 0.062,
  'group_w15': 0.0624,
  'group_w06': 0.0632,
  'group_w00': 0.0626},
 {'batch_id': 0,
  'realization': 1,
  'npv_function': 1886.010009765625,
  'group_w14': 0.0623,
  'group_w01': 0.0627,
  'group_w12': 0.0621,
  'group_w09': 0.0618,
  'group_w03': 0.0629,
  'group_w13': 0.0622,
  'group_w07': 0.0633,
  'group_w04': 0.063,
  'group_w02': 0.0628,
  'group_w10': 0.0619,
  'group_w08': 0.0617,
  'group_w05': 0.0631,
  'group_w11': 0.062,
  'group_w15': 0.0624,
  'group_w06': 0.0632,
  'group_w00': 0.0626}]

Objective results:

objectives_for_batch
Out[1]: 
shape: (1, 4)
┌───────────┬──────────┬───────────────────────┬──────────────┐
│ result_id ┆ batch_id ┆ total_objective_value ┆ npv_function │
│ ---       ┆ ---      ┆ ---                   ┆ ---          │
│ i64       ┆ i64      ┆ f64                   ┆ f64          │
╞═══════════╪══════════╪═══════════════════════╪══════════════╡
│ 0         ┆ 0        ┆ 1886.01001            ┆ 1886.01001   │
└───────────┴──────────┴───────────────────────┴──────────────┘
objectives_per_real
Out[2]: 
shape: (2, 19)
┌──────────┬───────────┬───────────┬───────────┬───┬───────────┬───────────┬───────────┬───────────┐
│ batch_id ┆ realizati ┆ npv_funct ┆ group_w14 ┆ … ┆ group_w11 ┆ group_w15 ┆ group_w06 ┆ group_w00 │
│ ---      ┆ on        ┆ ion       ┆ ---       ┆   ┆ ---       ┆ ---       ┆ ---       ┆ ---       │
│ i64      ┆ ---       ┆ ---       ┆ f64       ┆   ┆ f64       ┆ f64       ┆ f64       ┆ f64       │
│          ┆ i64       ┆ f64       ┆           ┆   ┆           ┆           ┆           ┆           │
╞══════════╪═══════════╪═══════════╪═══════════╪═══╪═══════════╪═══════════╪═══════════╪═══════════╡
│ 0        ┆ 0         ┆ 1886.0100 ┆ 0.0623    ┆ … ┆ 0.062     ┆ 0.0624    ┆ 0.0632    ┆ 0.0626    │
│          ┆           ┆ 1         ┆           ┆   ┆           ┆           ┆           ┆           │
│ 0        ┆ 1         ┆ 1886.0100 ┆ 0.0623    ┆ … ┆ 0.062     ┆ 0.0624    ┆ 0.0632    ┆ 0.0626    │
│          ┆           ┆ 1         ┆           ┆   ┆           ┆           ┆           ┆           │
└──────────┴───────────┴───────────┴───────────┴───┴───────────┴───────────┴───────────┴───────────┘
constraints_for_batch
Out[3]: 
shape: (1, 66)
┌───────────┬──────────┬───────────┬───────────┬───┬───────────┬───────────┬───────────┬───────────┐
│ result_id ┆ batch_id ┆ oil_prod_ ┆ oil_prod_ ┆ … ┆ oil_prod_ ┆ oil_prod_ ┆ oil_prod_ ┆ oil_prod_ │
│ ---       ┆ ---      ┆ rate_000  ┆ rate_001  ┆   ┆ rate_012. ┆ rate_013. ┆ rate_014. ┆ rate_015. │
│ i64       ┆ i64      ┆ ---       ┆ ---       ┆   ┆ scaled_vi ┆ scaled_vi ┆ scaled_vi ┆ scaled_vi │
│           ┆          ┆ f64       ┆ f64       ┆   ┆ ola…      ┆ ola…      ┆ ola…      ┆ ola…      │
│           ┆          ┆           ┆           ┆   ┆ ---       ┆ ---       ┆ ---       ┆ ---       │
│           ┆          ┆           ┆           ┆   ┆ f64       ┆ f64       ┆ f64       ┆ f64       │
╞═══════════╪══════════╪═══════════╪═══════════╪═══╪═══════════╪═══════════╪═══════════╪═══════════╡
│ 0         ┆ 0        ┆ 1000.0    ┆ 1000.0    ┆ … ┆ 0.133333  ┆ 0.133333  ┆ 0.133333  ┆ 0.133333  │
└───────────┴──────────┴───────────┴───────────┴───┴───────────┴───────────┴───────────┴───────────┘
constraints_per_real
Out[4]: 
shape: (2, 50)
┌──────────┬───────────┬───────────┬───────────┬───┬───────────┬───────────┬───────────┬───────────┐
│ batch_id ┆ realizati ┆ oil_prod_ ┆ oil_prod_ ┆ … ┆ group_w12 ┆ group_w13 ┆ group_w14 ┆ group_w15 │
│ ---      ┆ on        ┆ rate_000  ┆ rate_001  ┆   ┆ ---       ┆ ---       ┆ ---       ┆ ---       │
│ i64      ┆ ---       ┆ ---       ┆ ---       ┆   ┆ f64       ┆ f64       ┆ f64       ┆ f64       │
│          ┆ i64       ┆ f64       ┆ f64       ┆   ┆           ┆           ┆           ┆           │
╞══════════╪═══════════╪═══════════╪═══════════╪═══╪═══════════╪═══════════╪═══════════╪═══════════╡
│ 0        ┆ 0         ┆ 6000.0    ┆ 6000.0    ┆ … ┆ 0.0621    ┆ 0.0622    ┆ 0.0623    ┆ 0.0624    │
│ 0        ┆ 1         ┆ 6000.0    ┆ 6000.0    ┆ … ┆ 0.0621    ┆ 0.0622    ┆ 0.0623    ┆ 0.0624    │
└──────────┴───────────┴───────────┴───────────┴───┴───────────┴───────────┴───────────┴───────────┘

Gradient results:

obj_gradient_for_batch
Out[4]: 
shape: (1, 4)
┌───────────┬──────────┬──────────────┬────────────────────┐
│ result_id ┆ batch_id ┆ npv_function ┆ npv_function.total │
│ ---       ┆ ---      ┆ ---          ┆ ---                │
│ i64       ┆ i64      ┆ f64          ┆ f64                │
╞═══════════╪══════════╪══════════════╪════════════════════╡
│ 1         ┆ 1        ┆ -0.0         ┆ -0.0               │
└───────────┴──────────┴──────────────┴────────────────────┘
obj_gradient_perturbations
Out[5]: 
shape: (4, 21)
┌───────────┬──────────┬───────────┬───────────┬───┬───────────┬───────────┬───────────┬───────────┐
│ result_id ┆ batch_id ┆ realizati ┆ perturbat ┆ … ┆ group_w08 ┆ group_w07 ┆ group_w01 ┆ group_w05 │
│ ---       ┆ ---      ┆ on        ┆ ion       ┆   ┆ ---       ┆ ---       ┆ ---       ┆ ---       │
│ i64       ┆ i64      ┆ ---       ┆ ---       ┆   ┆ f64       ┆ f64       ┆ f64       ┆ f64       │
│           ┆          ┆ i64       ┆ i64       ┆   ┆           ┆           ┆           ┆           │
╞═══════════╪══════════╪═══════════╪═══════════╪═══╪═══════════╪═══════════╪═══════════╪═══════════╡
│ 1         ┆ 1        ┆ 1         ┆ 0         ┆ … ┆ 0.059048  ┆ 0.068553  ┆ 0.062976  ┆ 0.062975  │
│ 1         ┆ 1        ┆ 0         ┆ 1         ┆ … ┆ 0.061752  ┆ 0.069882  ┆ 0.065536  ┆ 0.061705  │
│ 1         ┆ 1        ┆ 0         ┆ 0         ┆ … ┆ 0.057099  ┆ 0.070677  ┆ 0.055112  ┆ 0.071082  │
│ 1         ┆ 1        ┆ 1         ┆ 1         ┆ … ┆ 0.064173  ┆ 0.066663  ┆ 0.068711  ┆ 0.062269  │
└───────────┴──────────┴───────────┴───────────┴───┴───────────┴───────────┴───────────┴───────────┘
constraint_gradient_for_batch
Out[6]: 
shape: (1, 33)
┌──────────┬───────────┬───────────┬───────────┬───┬───────────┬───────────┬───────────┬───────────┐
│ batch_id ┆ oil_prod_ ┆ oil_prod_ ┆ oil_prod_ ┆ … ┆ oil_prod_ ┆ oil_prod_ ┆ oil_prod_ ┆ oil_prod_ │
│ ---      ┆ rate_004  ┆ rate_003  ┆ rate_006  ┆   ┆ rate_014. ┆ rate_005. ┆ rate_012. ┆ rate_010. │
│ i64      ┆ ---       ┆ ---       ┆ ---       ┆   ┆ scaled    ┆ scaled    ┆ scaled    ┆ scaled    │
│          ┆ f64       ┆ f64       ┆ f64       ┆   ┆ ---       ┆ ---       ┆ ---       ┆ ---       │
│          ┆           ┆           ┆           ┆   ┆ f64       ┆ f64       ┆ f64       ┆ f64       │
╞══════════╪═══════════╪═══════════╪═══════════╪═══╪═══════════╪═══════════╪═══════════╪═══════════╡
│ 1        ┆ 0.0       ┆ 0.0       ┆ 0.0       ┆ … ┆ 0.0       ┆ 0.0       ┆ 0.0       ┆ 0.0       │
└──────────┴───────────┴───────────┴───────────┴───┴───────────┴───────────┴───────────┴───────────┘
constraint_gradient_perturbations
Out[7]: 
shape: (4, 51)
┌──────────┬───────────┬───────────┬───────────┬───┬───────────┬───────────┬───────────┬───────────┐
│ batch_id ┆ realizati ┆ perturbat ┆ oil_prod_ ┆ … ┆ group_w12 ┆ group_w13 ┆ group_w14 ┆ group_w15 │
│ ---      ┆ on        ┆ ion       ┆ rate_000  ┆   ┆ ---       ┆ ---       ┆ ---       ┆ ---       │
│ i64      ┆ ---       ┆ ---       ┆ ---       ┆   ┆ f64       ┆ f64       ┆ f64       ┆ f64       │
│          ┆ i64       ┆ i64       ┆ f64       ┆   ┆           ┆           ┆           ┆           │
╞══════════╪═══════════╪═══════════╪═══════════╪═══╪═══════════╪═══════════╪═══════════╪═══════════╡
│ 1        ┆ 0         ┆ 0         ┆ 6000.0    ┆ … ┆ 0.066225  ┆ 0.067566  ┆ 0.05967   ┆ 0.062201  │
│ 1        ┆ 0         ┆ 1         ┆ 6000.0    ┆ … ┆ 0.069046  ┆ 0.058626  ┆ 0.061307  ┆ 0.059147  │
│ 1        ┆ 1         ┆ 0         ┆ 6000.0    ┆ … ┆ 0.066712  ┆ 0.066355  ┆ 0.067376  ┆ 0.059584  │
│ 1        ┆ 1         ┆ 1         ┆ 6000.0    ┆ … ┆ 0.065746  ┆ 0.069017  ┆ 0.060187  ┆ 0.06393   │
└──────────┴───────────┴───────────┴───────────┴───┴───────────┴───────────┴───────────┴───────────┘

@codecov-commenter
Copy link

codecov-commenter commented Nov 6, 2024

Codecov Report

Attention: Patch coverage is 40.00000% with 3 lines in your changes missing coverage. Please review.

Project coverage is 90.72%. Comparing base (721f3c6) to head (f9c476a).

Files with missing lines Patch % Lines
src/ert/storage/local_ensemble.py 33.33% 2 Missing ⚠️
src/ert/run_models/everest_run_model.py 50.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #9161      +/-   ##
==========================================
+ Coverage   90.70%   90.72%   +0.02%     
==========================================
  Files         351      351              
  Lines       21903    21908       +5     
==========================================
+ Hits        19867    19876       +9     
+ Misses       2036     2032       -4     
Flag Coverage Δ
cli-tests 39.23% <20.00%> (+0.02%) ⬆️
gui-tests 71.73% <20.00%> (-0.04%) ⬇️
performance-tests 49.37% <20.00%> (-0.03%) ⬇️
unit-tests 79.63% <40.00%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@yngve-sk yngve-sk force-pushed the 24.10.25.store-everest-opt-results-in-ertstorage branch 14 times, most recently from 7081e58 to 32193bd Compare November 13, 2024 11:20
@yngve-sk yngve-sk changed the title (wip) Store everest results in ERT storage Store everest results in ERT storage Nov 13, 2024
@yngve-sk yngve-sk marked this pull request as ready for review November 13, 2024 11:26
@yngve-sk yngve-sk force-pushed the 24.10.25.store-everest-opt-results-in-ertstorage branch 13 times, most recently from 5b54ee7 to dda3db9 Compare November 15, 2024 13:06
@yngve-sk yngve-sk force-pushed the 24.10.25.store-everest-opt-results-in-ertstorage branch 7 times, most recently from 3b6d06d to 8583aa9 Compare November 25, 2024 07:48
@yngve-sk yngve-sk force-pushed the 24.10.25.store-everest-opt-results-in-ertstorage branch 3 times, most recently from 98b447a to 84d7248 Compare December 5, 2024 12:30
@yngve-sk yngve-sk force-pushed the 24.10.25.store-everest-opt-results-in-ertstorage branch from 84d7248 to 14b9e4a Compare December 10, 2024 08:43
@yngve-sk yngve-sk force-pushed the 24.10.25.store-everest-opt-results-in-ertstorage branch from 191fecc to db4c6dd Compare December 19, 2024 07:42
Copy link

codspeed-hq bot commented Dec 19, 2024

CodSpeed Performance Report

Merging #9161 will not alter performance

Comparing yngve-sk:24.10.25.store-everest-opt-results-in-ertstorage (08a476c) with main (ead6423)

Summary

✅ 24 untouched benchmarks

@yngve-sk yngve-sk force-pushed the 24.10.25.store-everest-opt-results-in-ertstorage branch 2 times, most recently from 3c74f6d to c694c16 Compare December 20, 2024 10:43
@yngve-sk yngve-sk force-pushed the 24.10.25.store-everest-opt-results-in-ertstorage branch 4 times, most recently from dfcba5a to 7f9f165 Compare December 20, 2024 12:24
@yngve-sk yngve-sk force-pushed the 24.10.25.store-everest-opt-results-in-ertstorage branch from 7f9f165 to 7edb72e Compare December 20, 2024 12:40
@yngve-sk yngve-sk force-pushed the 24.10.25.store-everest-opt-results-in-ertstorage branch from 7edb72e to 08a476c Compare December 20, 2024 12:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Refactor communication and storage of optimization results
3 participants